Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis.

Early recognition and treatment of sepsis are linked to improved patient outcomes. Machine learning-based early warning systems may reduce the time to recognition, but few systems have undergone clinical evaluation. In this prospective, multi-site cohort study, we examined the association between patient outcomes and provider interaction with a deployed sepsis alert system called the Targeted Real-time Early Warning System (TREWS). During the study, 590,736 patients were monitored by TREWS across five hospitals. We focused our analysis on 6,877 patients with sepsis who were identified by the alert before initiation of antibiotic therapy. Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within 3 h of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within 3 h. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk. Our findings indicate that early warning systems have the potential to identify sepsis patients early and improve patient outcomes and that sepsis patients who would benefit the most from early treatment can be identified and prioritized at the time of the alert.

Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing.

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.

Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system.

While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians' autonomy and support them across their entire workflow.

Impacts of an EMS Hospital Liaison Program on Ambulance Offload Times: A Preliminary Analysis.

INTRODUCTION: Ambulance patients who are unable to be quickly transferred to an emergency department (ED) bed represent a key contributing factor to ambulance offload delay (AOD). Emergency department crowding and associated AOD are exacerbated by multiple factors, including infectious disease outbreaks such as the coronavirus disease 2019 (COVID-19) pandemic. Initiatives to address AOD present an opportunity to streamline ambulance offload procedures while improving patient outcomes. STUDY OBJECTIVE: The goal of this study was to evaluate the initial outcomes and impact of a novel Emergency Medical Service (EMS)-based Hospital Liaison Program (HLP) on ambulance offload times (AOTs). METHODS: Ambulance offload times associated with EMS patients transported to a community hospital six months before and after HLP implementation were retrospectively analyzed using proportional significance tests, t-tests, and multiple regression analysis. RESULTS: A proportional increase in incidents in the zero to <30 minutes time category after program implementation (+2.96%; P <.01) and a commensurate decrease in the proportion of incidents in the 30 to <60 minutes category (-2.65%; P <.01) were seen. The fully adjusted regression model showed AOT was 16.31% lower (P <.001) after HLP program implementation, holding all other variables constant. CONCLUSION: The HLP is an innovative initiative that constitutes a novel pathway for EMS and hospital systems to synergistically enhance ambulance offload procedures. The greatest effect was demonstrated in patients exhibiting potentially life-threatening symptoms, with a reduction of approximately three minutes. While small, this outcome was a statistically significant decrease from the pre-intervention period. Ultimately, the HLP represents an additional strategy to complement existing approaches to mitigate AOD.